by Kate Walsh and Sandi Jacobs

It's January, so it must be time for Quality Counts, Education Week's annual report card on the quality of states' education policies. The 2008 edition includes a spanking new set of teacher quality indicators used to rate states, ranging from teacher salaries to professional development. Last year's QC left out any ratings for states in the area of teacher quality, after its editors lost confidence in the indicators they had been using and decided to go back to the drawing board.

Do the new indicators get any closer to what matters? Mostly yes, but there are a few notable clunkers. We certainly have our own opinions about how states could address teacher quality--spelled out in our State Teacher Policy Yearbook--but our issue with Quality Counts is more than a territorial claim.

To its credit, Quality Counts takes a fairly progressive view of the teaching profession, as well as broadening the scope to include such important factors as school leadership. They have also added a great indicator giving states credit for reducing a new teachers' workload.

At the other end of the spectrum are a number of indicators that appear to suggest a state is serious about tackling teacher quality, but which actually have little impact on students in the classroom. State-led professional development efforts, which tend to embody all the worst things about ineffective professional development, fall into that category.

Another is an absolute jaw-dropper for its inclusion as an indicator: class size reduction.

There is simply no evidence that state-funded class size reduction initiatives help teacher quality. For starters, the country does not have enough teachers of the quality needed to staff the current number of classrooms. Even if teachers were available to staff a classroom reduction initiative, states would have to provide staggering amounts of funding to bring classrooms down to a small enough size to make a difference. Much has been documented about California's infamous class size reduction initiative, which led to statewide teacher shortages and declining student test scores.

There's certainly no research basis for QC's insertion of class size into the teacher quality equation. McKinsey and Company's recent report, drawing on the work of Eric Hanushek and others, noted that of 112 studies of the impact of reduced class sizes, only 9 found a positive relationship, and none of these effects was substantial.

Quality Counts errs as well by giving states credit when none is deserved. For instance, QC credits 39 states with requiring basic skills tests of prospective teachers. Yes, these states do technically have this requirement, but well over half of these states do not make this test a condition when it is needed, that is, when the candidate applies to ed school. The consequence is that ed schools routinely admit substandard candidates. They then invest valuable classroom time and taxpayer dollars to prepare them to pass a math and reading test that a middle school student should be able to pass. The practice borders on the negligent and is unique to this country. It is certainly not something for which to applaud a state.

Similarly, Quality Counts gives credit to 47 states for having an alternative certification program. Yet the routes in more than half of these states are in every respect no different than what is required for traditional certification. Even more troubling is the practice of states renaming their emergency license as an "alternative" license in order to comply with NCLB requirements. It's a level of gaming the system that no one should tolerate. By our admittedly tough count, only 6 states offer a genuine alternate route, but even allowing a loose definition to apply, only 30 states offer one that isn't actually an emergency license in disguise.

Much of the data reported in QC is collected by a rather risky method, in which officials in the states' departments of education answer a survey. While QC editors claim also to require documentation, the published results reveal the inherent limitations that come with relying on states to provide a meaningful answer. For example, 38 states say they grant reciprocity to teachers coming from out of state. Serving as the supplied documentation that QC's editors require, states would likely have produced the interstate agreement known as the NASDTEC agreement. It isn't until you peel back the terms of this so-called reciprocity agreement that you find that most states aren't really waiving any requirements, even for teachers who earned a traditional license in another state. It's a meaningless system that makes states appear flexible but which in fact sends teachers jumping through all sorts of hoops.

In the interest of full disclosure, we will admit to feeling somewhat taken aback by the notable absence of any acknowledgment of the Yearbook. The omission of an acknowledgment of our contribution to this area strikes us (and others such as Fordham's Gadfly), at the very least, as unsportsmanlike. While we'll certainly get over our bruised feelings, the bottom line is that we are much more concerned about the need to hold states firmly accountable for their role in shaping the quality of the nation's teaching force. There's too much at stake not to do this right.